Stitching method and apparatus for panoramic stereo video system

ABSTRACT

A method of generating a stereoscopic panorama is provided, the method comprising: processing a first right image, a second right image, a first left image, and a second left image to derive a right homography between the first right image and the second right image and a left homography between the first left image and the second left image; stitching the first right image with the second right image to generate a right panorama, the first left image with the second left image to generate a left panorama wherein the right homography is consistent with the left homography; and generating a stereo panorama using the right panorama and the left panorama.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Phase application under 35 U.S.C. § 371of PCT Application No. PCT/CN2016/070823, filed Jan. 13, 2016, whichapplication claims priority benefit to PCT Application No.PCT/CN2016/070712, filed Jan. 12, 2016. The entire contents of all ofthe above-referenced applications are hereby incorporated herein byreference.

TECHNICAL FIELD

The present invention relates to panoramic stereo video systems forcapturing, processing, compressing, and displaying 3D panoramic stereovideos, and more particularly, to methods and apparatus for stitchingimages to generate 3D panoramic stereo videos in the panoramic stereovideo systems.

BACKGROUND

In panoramic stereo video systems, a multi-camera array with certaingeometric configuration is often used to capture images. For example,the camera array may have 8 pairs of cameras installed on a mountingframe in the shape of a regular octagon, with each pair of camerasplaced on each side of the octagon and having parallel optical axes.Such camera array can capture eight pairs of video data, with each paircapturing two videos to generate a stereo video. Two panoramic videoscan be generated by stitching the videos captured by all the leftcameras and all the right cameras in the eight pairs respectively. Thetwo panoramic videos can then be sent to a display system to display apanoramic stereo video.

As compared with traditional single panorama stitching, stereo stitchingfaces more constraints. In particular, since the camera array often hasa particular geometric configuration, the video data captured by thevarious cameras in the camera array will often have certain relationshipamong themselves that must be taken into consideration. Therefore, thereis a need for a new method of stitching images to generate a stereopanoramic video that optimizes performance while satisfying certainconstraints.

SUMMARY OF THE INVENTION

To address issues in the prior art, embodiments of the present inventionprovide a method and apparatus for stitching images to generate a stereopanoramic video that optimizes performance while satisfying certainconstraints.

In accordance with an embodiment of the present invention, an imageacquisition and processing apparatus for capturing images to generate astereoscopic panorama is provided, the image acquisition and processingapparatus comprising: a camera array comprising a first right camera, asecond right camera, a first left camera, and a second left camera,wherein a relative position between the first right camera and thesecond right camera is substantially the same as a relative positionbetween the first left camera and the second left camera; and an imageprocessing unit configured to stitch a first right image captured by thefirst right camera with a second right image captured by the secondright camera to generate a right panorama, and to stitch a first leftimage captured by the first left camera with a second left imagecaptured by the second left camera to generate a left panorama, whereinthe image processing unit is configured to derive a right homographybetween the first right image and the second right image consistent withleft a homography between the first left image and the second leftimage.

Preferably, the image processing unit is configured to identify andmatch a feature in the first right image and the second right image.

Preferably, the image processing unit is configured to estimate theright homography using a random sample consensus (RANSAC) algorithm.

Preferably, the first right camera and the second right camera compriseparallel optical axes.

Preferably, the first right camera, the second right camera, the firstleft camera, and the second left camera are situated substantially on aplane.

Preferably, the first right camera and the second right camera aresituated on a first side of a regular polygon, and the first left cameraand the second left camera are situated on a second side of the regularpolygon adjacent to the first side.

In accordance with another embodiment of the present invention, a methodof generating a stereoscopic panorama is provided, the methodcomprising: processing a first right image, a second right image, afirst left image, and a second left imaging to derive a right homographybetween the first right image and the second right image and ahomography between the first left image and the second left image;stitching the first right image with the second right image to generatea right panorama, the first left image with the second left image togenerate a left panorama wherein the right homography is consistent withthe left homography; and generating a stereo panorama using the rightpanorama and the left panorama.

Preferably, the method further comprises identifying and matching afeature in the first right image and the second right image.

Preferably, the method further comprises estimating the right homographyusing a random sample consensus (RANSAC) algorithm.

Preferably, the method further comprises capturing the first rightimage, the second right image, the first left image, and the second leftimaging using a camera array comprising a first right camera, a secondright camera, a first left camera, and a second left camera, wherein arelative position between the first right camera and the second rightcamera is substantially the same as a relative position between thefirst left camera and the second left camera, and the first right cameraand the second right camera comprise parallel optical axes.

Preferably, the first right camera, the second right camera, the firstleft camera, and the second left camera are situated substantially on aplane.

Preferably, the first right camera and the second right camera aresituated on a first side of a regular polygon, and the first left cameraand the second left camera are situated on a second side of the regularpolygon adjacent to the first side.

In accordance with embodiments of the present invention, images arecaptured using a camera array having certain geometric configuration,and images captured by the camera array are stitched to generate astereo panoramic video using a method that optimizes performance whilesatisfying certain constraints.

BRIEF DESCRIPTION OF THE DRAWINGS

To better illustrate the technical features of the embodiments of thepresent invention, various embodiments of the present invention will bebriefly described in conjunction with the accompanying drawings.

FIG. 1 is an exemplary schematic diagram for a panoramic stereo videosystem in accordance with an embodiment of the present invention.

FIG. 2 is an exemplary schematic diagram for a camera array in thepanoramic stereo video system in accordance with an embodiment of thepresent invention.

FIG. 3 is an exemplary schematic diagram for a data processing unit in apanoramic stereo video system in accordance with an embodiment of thepresent invention.

FIG. 4 is an exemplary flowchart for a method of stitching a panoramicstereo video in accordance with an embodiment of the present invention.

FIG. 5 is an exemplary flowchart for a method of displaying a panoramicstereo video in accordance with an embodiment of the present invention.

FIG. 6 is an exemplary flowchart for a method of generating astereoscopic panorama in accordance with an embodiment of the presentinvention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

To better illustrate the purpose, technical feature, and advantages ofthe embodiments of the present invention, various embodiments of thepresent invention will be further described in conjunction with theaccompanying drawings. It is obvious that the draws are but forexemplary embodiments of the present invention, and that a person ofordinary skill in the art may derive additional draws without deviatingfrom the principles of the present invention.

In accordance with embodiments of the present invention, a panoramicstereo video system including multi-camera video capturing, dataprocessing, stereo video coding, transmission, and 3D displaying isprovided. The panoramic stereo video system employs real-time multi-viewvideos capturing, image rectification and pre-processing, and region ofinterest (ROI) based stereo video compression. After the transmissionand decoding process, a head-mounted display (HMD) headset is used todisplay the left and right views.

1. System Overview

FIG. 1 is an exemplary schematic diagram for a panoramic stereo videosystem in accordance with an embodiment of the present invention. Thepanoramic stereo video system 100 uses a camera array to capture 3Dpanoramic video, and displays the captured 3D panoramic video on eithera 3D TV or a head-mounted virtual reality display device. As shown inFIG. 1, the panoramic stereo video system 100 includes a data capturingunit 200, a data processing unit 300, and a data display unit 400. Thedata capturing unit 200 includes a plurality of cameras in a cameraarray 210, and a camera calibration unit 220. The data processing unit300 includes a data pre-processing unit 310 and an advanced stereo videotranscoding unit 320. The data display unit 400 includes a decoding unit410, and a display headset 420.

2. Data Capturing Unit

As shown in FIG. 1, the data capturing unit 200 includes a plurality ofcameras in a camera array 210, and a camera calibration unit 220 forcalibrating the camera array 210.

2.1. Camera Array

FIG. 2 is an exemplary schematic diagram for a camera array in thepanoramic stereo video system in accordance with an embodiment of thepresent invention.

As shown in FIG. 2, the camera array 210 has 16 high-definition camerasc1-c16 installed on a mounting frame in the shape of a regular octagon,with a pair of cameras placed on each side of the octagon. The twocameras on each side, such as c1 and c2, have parallel optical axes, andare separated by a distance d. The raw video data captured by the cameraarray 210 is transmitted through cable to a computer for furtherprocessing. The parameters for the camera are listed in Table 1 below.

TABLE 1 Sensor OV2710 Sensor Size 1/2.7 inch Pixel Size 3 * 3 Image Area5856 * 3276 Resolution FULL HD 1920(H) * 1080(V) Frame MJPEG@30fps USBprotocol USB2.0 HS/FS AEC Support AEB Support AGC Support Lens Standard2.1 mm, Parameter optional/2.5/2.8/3.6/6 mm/FOV(D)170 Degree/187 Degree

It should be noted that while the camera array is depicted as a regularoctagon in FIG. 2, the camera array can be configured in other shapes inaccordance with other embodiments of the present invention.Specifically, in one embodiment of the present invention, the camerasare installed on a rigid frame, so that the relative positions among theplurality of cameras are substantially constant. In another embodimentof the present invention, the cameras are located substantially on thesame plane, such as on the sides of a polygon.

2.2. Camera Calibration

To stitch images captured by the cameras together and generate 3Deffects, it is necessary to obtain both the internal and parameters ofthe cameras. The extrinsic parameters include the rotation andtranslation among the cameras, so that the images captured by differentcameras can be rectified and aligned in the horizontal direction. Also,there may be distortions in the images captured by the cameras, and toobtain images free of the distortions, it is necessary to know thecamera distortion parameters. These parameters are obtained during thecamera calibration process.

2.2.1. Calibration of Internal and Distortion Parameters

The internal and distortion parameters of the cameras can be obtainedusing a variety of methods, such as the calibration method proposed byZhengyou Zhang, and tools like MatLab can be employed to obtain theseparameters.

2.2.2. Calibration of Extrinsic Parameters

After obtaining the internal parameters of the cameras, a method basedon structure from motion is employed to obtain the rotations andtranslations among the cameras. This method has the followingadvantages.

Efficiency: there is no need to calibrate the cameras pair by pair.Rather, all cameras capture videos of a scene simultaneously duringcalibration, and the extrinsic parameters for all the cameras can beobtained at the same time.

Accuracy: in pattern-based calibration methods, the pattern needs to becaptured by two adjacent cameras, which often results in lowerresolution for the pattern and reduces the accuracy of the calibration.In our structure from motion based method, the movement of each camerawill be estimated independently to obtain the parameters, and there isno need for adjacent cameras to have an overlapping field of view. Thus,we can place the cameras closer to the scene to be captured to achievebetter accuracy.

Extensibility: since our method does not require an overlapping field ofview for adjacent cameras, it is even applicable to cameras placed inback-to-back positions.

2.3. Data Capturing Method

Data from the 16 cameras are captured and stored using software, andthen provided to the data processing unit. Image data for each framecaptured by the cameras can be collected using software such as FFmpegand DirectShow (or DShow). The frames captured by each camera arecompressed and then stored as video files. Since there are a number ofcameras, the frames captured by the cameras need to be synchronized,such as using time stamps. For example, each frame captured by thecameras can be affixed with a time stamp, and placed in a queue so thatit is synchronized with other frames with the same time stamp. Thesynchronized frames are coded into video streams, and stored locally ortransmitted through the network together.

3. Data Processing Unit

As shown in FIG. 1, the data processing unit 300 includes datapre-processing unit 310 and an advanced stereo video transcoding unit320.

FIG. 3 is an exemplary schematic diagram for a data processing unit in apanoramic stereo video system in accordance with an embodiment of thepresent invention. As shown in FIG. 3, the data pre-processing unit 310includes a timeline synchronization 311 for synchronizing the imagescaptured by the cameras, a number of decoders 312 for decoding streamsof raw video, a number of rectifiers 313 for rectifying the raw video,an encoder 314 for video processing, including noise reducing andediting, a stitching unit for stitching videos to generate panoramicvideo. The data pre-processing unit 310 outs a left view video and aright view video to the advanced stereo video transcoding unit 320. Theadvanced stereo video transcoding unit 320 generates a motion map 321and a texture map 322 for the video, and a hybrid region of interest(ROI) generating unit 323 identifies regions of interests in the videosbased on the motion map and 321 the texture map 322. A bits allocationunit 324 allocates bits based on the identified region of interests, anda HEVC encoding unit 325 encodes the videos. A H.265 packetizer 326packs the encoded videos for transmission.

FIG. 4 is an exemplary flowchart for a method of stitching a panoramicstereo video in accordance with an embodiment of the present invention.

3.1 Distortion Correction and Preprocessing

The frames captured by the cameras are warped in accordance with thedistortion parameters obtained during the calibration process to obtainframes free of distortions. In order to enhance the accuracy of imagealignment and stitching, the frames are filtered first to reduce noises.

3.2. Image Alignment

Image alignment is performed on each pair of cameras located on eachside of the octagon, and the images captured by each pairs of camerasare aligned in the horizontal direction. In accordance with anembodiment of the present invention, each frame captured by the pairs ofcameras is warped to a plane that is parallel to the optical axes of thepair of cameras.

4. Panoramic Video Stitching

The camera array has 8 pairs of cameras. The frames captured by all theleft cameras are projected onto a cylinder, and then stitched into apanoramic image. A panoramic video can be obtained by repeating theabove steps for all the frames captured by the left cameras. Anotherpanoramic video can be obtained by processing the frames captured by theright cameras in the same fashion. These two panoramic videos form apanoramic stereo video.

5. Data Display Unit

As shown in FIG. 1, the data display unit 400 includes a decoding unit410, and a display headset 420. After going through an encoding anddecoding system, the panoramic stereo video is played on display headset420, which can be a wearable virtual reality (VR) equipment, such as oneprovided by Oculus VR. The panoramic stereo video is renderedrespectively to the left display and the right display of the Oculusdevice. The display area of the panoramic stereo video can be adjustedin accordance with the movement of the detection device to simulate thechange in perspective in the virtual reality.

FIG. 5 is an exemplary flowchart for a method of displaying a panoramicstereo video in accordance with an embodiment of the present invention.As shown in FIG. 5, in step 501, the encoded video stream is firstdecoded to YUV. In step 502, position calculation and view selection aremade in accordance with the Oculus sensors data. In step 503, the imagesfor the left eye and the right eye are respectively rendered. In step504, the rendered images are displayed on the Oculus display headset.

6. Stereo Stitching Method

Stereo stitching is different from traditional single panoramastitching. Several specific problems for stereo stitching are listed asfollows:

-   -   1) each pair of stereo images should be rectified.    -   2) disparity of different stereo images are different so that        disparity preservation should be considered.    -   3) homography of two overlapping images captured by each left        cameras in FIG. 1 is related to homography of their        corresponding stereo images captured by each right cameras.

FIG. 6 is an exemplary flowchart for a method of generating astereoscopic panorama in accordance with an embodiment of the presentinvention. As shown in FIG. 6, the method includes the following steps.

6.1. Image Capturing and Preprocessing

Step 601: capturing a first right image, a second right image, a firstleft image, and a second left image using a camera array comprising afirst right camera, a second right camera, a first left camera, and asecond left camera. Here, the camera array may have the configuration inFIG. 2, and have eight pairs of cameras on the sides of a regularoctagon. Each pair of cameras have parallel optical axes. Specifically,the relative position among the right cameras is substantially the sameas the relative position among the left cameras. As a result, the imagescaptured by the right cameras are related to the images captured by theleft cameras in a specific way.

Although the cameras on the camera array are located in a small range,the light condition of different frames can be very different, as someof the cameras are facing opposite directions as other cameras. Toenhance robustness in the estimation and calibration of cameraparameters, exposure compensation is performed. To speed up processing,resizing of the images is also performed in preprocessing.

6.2. Image Matching

Step 602: identifying and matching a feature in the first right imageand the second right image.

Feature detection and matching are the basic steps in the stitchingmethod. In one embodiment, SURF features are used to find matchingimages. Since only a small number of matching features need to be foundin the adjacent images, image matching can be performed efficiently.

6.3. Homography Estimation

Step 603: processing the first right image, the second right image, thefirst left image, and the second left imaging to derive a righthomography between the first right image and the second right image anda left homography between the first left image and the second leftimage.

As discussed above, stereo stitching faces more constraints thantraditional single panorama stitching. A new optimization method is usedto provide a solution while satisfying certain constraints.

Suppose x_(l), x_(r) are two frames captured by one pair of cameras (oneleft camera and one right camera as shown in FIG. 2), y_(l), y_(r) aretwo frames captured by the adjacent pair of cameras. Moreover, x_(l)overlaps with y_(l), and x_(r) overlaps with y_(r). To generatepanoramas, it is needed to stitch x_(l) with y_(l), and x_(r) withy_(r). In one embodiment, the homography H_(l) between x_(l) and y_(l)is kept consistent with the homography H_(r) between x_(r) and y_(r). Ina preferred embodiment, H_(l) and H_(r) are the same, denoted as H.Equation (1) below shows the re-projection error from x_(l) to y_(l),where Ω indicates the set of matching features between x_(l) and y_(l).Similarly, the re-projection error between x_(r) and y_(r) is calculatedas in equation (2).

$\begin{matrix}{E_{l} = {\sum\limits_{{({{\overset{\sim}{x}}_{l},{\overset{\sim}{y}}_{l}})} \in \Omega}^{\;}{{{H{\overset{\sim}{x}}_{l}} - {\overset{\sim}{y}}_{l}}}^{2}}} & (1) \\{E_{r} = {\sum\limits_{{({{\overset{\sim}{x}}_{r},{\overset{\sim}{y}}_{r}})} \in \Omega}^{\;}{{{H{\overset{\sim}{x}}_{r}} - {\overset{\sim}{y}}_{r}}}^{2}}} & (2)\end{matrix}$

In one embodiment, one more constraint is used to ensure the consistenceof the homography. Since the purpose of stereo stitching is to generatetwo stereo panoramas, horizontal disparity should be considered whencalculating homography. As indicated by Equation (3) below, to obtainthe ideal homography, only horizontal disparity should be encouraged,while vertical disparity resulting from cumulative error and outliers infeature matching should be suppressed, where D is the set includingmatching feature points between one pair of frames (captured by oneleft-right pair of cameras), E_(v) is the orthogonal projection onto thevertical direction (denoted as y axis).

$\begin{matrix}{E_{v} = {\sum\limits_{{({{\overset{\sim}{x}}_{l},{\overset{\sim}{x}}_{r}})} \in D}^{\;}{{{H{\overset{\sim}{x}}_{l}} - {H{\overset{\sim}{x}}_{r}}}}^{2}}} & (3)\end{matrix}$

The overall optimization problem is formalized as formula (4), where anextra parameter λ is introduced as a hyper parameter to balance the twoparts.min(E _(l) +E _(r))+λE _(v)  (4)

In one embodiment, RANSAC is used to estimate a robust solution ofhomography between each two pair of overlapping images, which is a basicroutine for reducing the influence of outliers thus and enhancingrobustness.

6.4 Image Composition

Step 604: stitching the first right image with the second right image togenerate a right panorama, the first left image with the second leftimage to generate a left panorama wherein the right homography isconsistent with the left homography. Preferably, the right homography isthe same as the left homography.

Once the homography is estimated, two pairs of images can be warped andblended. In one embodiment, conventional warping-seam finding-blendingroutines are used with good performance. Color based graph cut algorithmfor seam finding can be implemented, and multi-band blending isperformed to generate the final panoramas.

Step 605: generating a stereo panorama using the right panorama and theleft panorama. Finally, two panoramas are warped using cylindricalcoordinate, and a stereo panorama is generated.

In accordance with embodiments of the present invention, images arecaptured using a camera array having certain geometric configuration,and images captured by the camera array are stitched to generate astereo panoramic video using a method that optimizes performance whilesatisfying certain constraints.

The various modules, units, and components described above can beimplemented as an Application Specific Integrated Circuit (ASIC); anelectronic circuit; a combinational logic circuit; a field programmablegate array (FPGA); a processor (shared, dedicated, or group) thatexecutes code; or other suitable hardware components that provide thedescribed functionality. The processor can be a microprocessor providedby from Intel, or a mainframe computer provided by IBM.

Note that one or more of the functions described above can be performedby software or firmware stored in memory and executed by a processor, orstored in program storage and executed by a processor. The software orfirmware can also be stored and/or transported within anycomputer-readable medium for use by or in connection with an instructionexecution system, apparatus, or device, such as a computer-based system,processor-containing system, or other system that can fetch theinstructions from the instruction execution system, apparatus, or deviceand execute the instructions. In the context of this document, a“computer-readable medium” can be any medium that can contain or storethe program for use by or in connection with the instruction executionsystem, apparatus, or device. The computer readable medium can include,but is not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus or device,a portable computer diskette (magnetic), a random access memory (RAM)(magnetic), a read-only memory (ROM) (magnetic), an erasableprogrammable read-only memory (EPROM) (magnetic), a portable opticaldisc such a CD, CD-R, CD-RW, DVD, DVD-R, or DVD-RW, or flash memory suchas compact flash cards, secured digital cards, USB memory devices,memory sticks, and the like.

The various embodiments of the present invention are merely preferredembodiments, and are not intended to limit the scope of the presentinvention, which includes any modification, equivalent, or improvementthat does not depart from the spirit and principles of the presentinvention.

The invention claimed is:
 1. An image acquisition and processingapparatus for capturing images to generate a stereoscopic panorama, theimage acquisition and processing apparatus comprising: a camera arraycomprising a first right camera, a second right camera, a first leftcamera, and a second left camera, wherein a relative position betweenthe first right camera and the second right camera is substantially thesame as a relative position between the first left camera and the secondleft camera; and an image processing unit configured to stitch a firstright image captured by the first right camera with a second right imagecaptured by the second right camera to generate a right panorama, and tostitch a first left image captured by the first left camera with asecond left image captured by the second left camera to generate a leftpanorama, wherein the image processing unit is configured to derive aright homography between the first right image and the second rightimage consistent with a left homography between the first left image andthe second left image, wherein the right homography is the same as theleft homography.
 2. The image acquisition and processing apparatus ofclaim 1, wherein the image processing unit is configured to rectify thefirst right image and the first left image in a first direction.
 3. Theimage acquisition and processing apparatus of claim 2, wherein the imageprocessing unit is configured to warp the first right image and thefirst left image to a plane parallel to the optical axes of the firstright camera and the first left camera.
 4. The image acquisition andprocessing apparatus of claim 2, wherein the image processing unit isconfigured to suppress disparity in a second direction perpendicular tothe first direction.
 5. The image acquisition and processing apparatusof claim 1, wherein the image processing unit is configured to identifyand match a feature in the first right image and the second right image.6. The image acquisition and processing apparatus of claim 1, whereinthe image processing unit is configured to estimate the right homographyusing a random sample consensus (RANSAC) algorithm.
 7. The imageacquisition and processing apparatus of claim 1, wherein the first rightcamera and the second right camera comprise parallel optical axes. 8.The image acquisition and processing apparatus of claim 1, wherein thefirst right camera, the second right camera, the first left camera, andthe second left camera are situated substantially on a plane.
 9. Theimage acquisition and processing apparatus of claim 8, wherein the firstright camera and the second right camera are situated on a first side ofa regular polygon, and the first left camera and the second left cameraare situated on a second side of the regular polygon adjacent to thefirst side.
 10. A method of generating a stereoscopic panorama, themethod comprising: processing a first right image, a second right image,a first left image, and a second left image to derive a right homographybetween the first right image and the second right image and a lefthomography between the first left image and the second left image;stitching the first right image with the second right image to generatea right panorama, the first left image with the second left image togenerate a left panorama wherein the right homography is consistent withthe left homography, wherein the right homography is the same as theleft homography; and generating a stereo panorama using the rightpanorama and the left panorama.
 11. The method of claim 10, furthercomprising: rectifying the first right image and the first left image ina first direction.
 12. The method of claim 11, further comprising:warping the first right image and the first left image to a plane. 13.The method of claim 11, further comprising: suppressing disparity in asecond direction perpendicular to the first direction.
 14. The method ofclaim 11, further comprising: identifying and matching a feature in thefirst right image and the second right image.
 15. The method of claim10, further comprising: estimating the right homography using a randomsample consensus (RANSAC) algorithm.
 16. The method of claim 10, furthercomprising: capturing the first right image, the second right image, thefirst left image, and the second left image using a camera arraycomprising a first right camera, a second right camera, a first leftcamera, and a second left camera, wherein a relative position betweenthe first right camera and the second right camera is substantially thesame as a relative position between the first left camera and the secondleft camera, and the first right camera and the second right cameracomprise parallel optical axes.
 17. The method of claim 16, wherein thefirst right camera, the second right camera, the first left camera, andthe second left camera are situated substantially on a plane.
 18. Themethod of claim 17, wherein the first right camera and the second rightcamera are situated on a first side of a regular polygon, and the firstleft camera and the second left camera are situated on a second side ofthe regular polygon adjacent to the first side.